1. Field of the Invention
The present invention relates generally to image sensing apparatuses, and more specifically, to multi-picture frame processing function.
2. Description of the Related Art
Among various image sensing apparatuses presently commercially available include a video camera with a recording function (a camcorder), a vide camera without any recording function, etc., any of these has only one system of circuitry for sensing the image of an object.
FIG. 7 is a block diagram schematically showing the entire structure of a conventional video camera. Referring to FIG. 7, light reflected from an object (not shown) is taken up into the video camera by lenses 11. The light taken up by lenses 11 forms the optical image of the object on the light receiving surface of an image sensing device 12 through the opening of an iris 17. Image sensing device 12 is formed of a CCD (Charge Coupled Device), etc. Image sensing device 12 converts the optical image formed on the light receiving surface into an electrical signal and applies the signal to a CDS (Correlation Double Sampling) circuit 13. CDS circuits 13 restrains low band noise included in the electrical signal applied from image sensing device 12. The electrical signal having its low band noise restrained by CDS circuit 13 is applied to an AGC (Automatic Gain Control) amplifier 14 and an iris control circuit 16.
AGC amplifier 14 amplifies or attenuates the electrical signal from CDS circuit 13 under the control of AGC control circuit, and controls the signal at a prescribed level suitable for processing in signal processing circuitry in succeeding stages. More specifically, AGC amplifier 14 is-a variable gain amplifier, while AGC control circuit 15 outputs a control signal for controlling the gain of AGC amplifier 14. AGC control circuit 15 responds to an output signal from AGC amplifier 14 and outputs a control signal in accordance with the difference between the output signal level of AGC amplifier 14 and the above stated prescribed level. If the output signal level of AGC amplifier 14 is lower than the prescribed level, AGC control circuit 15 controls AGC amplifier 14 so as to increase its gain, and if the output signal level of AGC amplifier 14 is higher than the prescribed level, AGC control circuit 15 controls AGC amplifier 14 so as to lower its gain. Consequently, the output signal level of AGC amplifier 14 is kept constantly at the prescribed level.
Iris control circuit 16 adjust the diameter of the opening of iris 17 so that the output signal level of CDS circuit 13 is always constant. When the opening diameter of iris 17 becomes larger, the amount of incident light upon the light receiving surface of image sensing device 12 increases, and, therefore, the output level of CDS circuit 13 rises. Conversely, when the opening diameter of iris 17 becomes smaller, the amount of incident light upon the light receiving surface of image sensing device 12 decreases, and, therefore, the output signal level of CDS circuit 13 decreases. Iris control circuit 16 therefore operates to reduce the opening diameter of iris 17 when the output signal level of CDS circuit 13 becomes higher than a certain reference level, while operates to increase the opening diameter of iris when the output signal level of CDS circuit 13 becomes lower than the reference level. Thus, the amount of incident light upon the light receiving surface of image sensing device 12 is always kept substantially constant, and, therefore, the output signal level of CDS circuit 13 is always kept substantially constant.
The electrical signal having its level controlled by AGC amplifier 14 is applied to an LPF (Low Pass Filter) 18 and a pixel separating circuit 110. LPF 18 extracts the low frequency component of the electrical signal applied from AGC amplifier 14. The low frequency component extracted by LPF 18 undergoes a prescribed processing by a Y signal processing circuit 19 and becomes a luminance signal Y representing the brightness of the above stated optical image.
Pixel separating circuit 110 separates the electrical signal applied from AGC amplifier 14 as for every pixel on the light receiving surface of image sensing device 12, and produces two kinds of signals containing different kinds of color information. These two kinds of signals produced by pixel separating circuit 110 are subjected to a prescribed operation by a matrix circuit 111 and converted into three primary color signals R, G and B.
These primary color signals R, G and B are input into a white balance circuit 116. White balance circuit 116 controls the gains of the primary color signals R, G and B so that white balance is made.
Generally, a color sensing apparatus such as color video camera possesses white balance function for reproducing the color of an object recognized by human eyes on a picture plane.
Human eyes recognize a white object as white regardless of its change as long as the color temperature of light illuminating the object to be seen is within a certain range. However, the higher the color temperature of light illuminating is, the more bluish the color of the object recognized by the image sensing device becomes while the lower the color temperature of the light illuminating, the more reddish the color of the object recognized by the device becomes. In other words, the color recognized by the image sensing device and the color recognized by human eyes are different depending upon the color temperatures of illuminating light.
Therefore, a color image sensing apparatus is provided with white balancing function which automatically adds a correction to color information obtained by sensing the image of an object depending upon the color temperature of the light illuminating the object. In the image sensing apparatus shown in FIG. 7, the white balancing function is implemented by white balance circuit 116.
Matrix circuit 112 further subjects the primary color signals R, G and B having their gains controlled by white balance circuit 116 to a prescribed operation and produces two color difference signals R - Y and B - Y.
More specifically, white balance circuit 116 corrects the gains of the primary color signals R, G and B from matrix circuit 111 so that the average level of each of the color difference signals R - Y and B - Y for one picture plane becomes zero or a prescribed value close to zero.
A white object can thus be reproduced white on a picture plane regardless of the color temperature of light illuminating by controlling the gains of the primary color signals so that the average levels of the color difference signals R - Y and B - Y approximately equal to the value (0) corresponding to "WHITE".
Which value the average level of each of the color difference signals R - Y and B - Y for one picture plane takes by controlling the gains of the primary color signals from R, Y and B by white balance circuit 116 is predetermined in accordance with the spectroscopic characteristics of lenses 11 and image sensing device 12, and white balance circuit 116 is designed depending upon the value.
The color difference signals R - Y and B - Y produced by matrix circuit 112 is applied to an encoder circuit 113. Encoder circuit 113 balanced modulates these color difference signals R - Y and B - Y with a sub-carrier wave of 3.58 MHz and produces a chroma signal C.
The luminance signal Y and chroma signal C produced by Y signal processing circuit 19 and encoder circuit 113, respectively are mixed by an adder 114. A signal obtained by this mixing is the video signal of the above-stated optical image. The video signal is externally output through an output circuit 115.
In Y signal processing circuit 19, various processing such as gamma correction, blanking processing, contour compensation processing, white clipping, set up processing, and mixing of synchronizing signals are conducted.
A timing generator 240 generates timing pulses for controlling the operation timings of image sensing device 12 and CDS circuit 13. A synchronizing signal generator 250 generates synchronizing signals for controlling the operation timings of the internal-circuits in automatic gain amplifier 14 and video signal producing portion 300. Timing generator 240 and synchronizing signal generator 250 are controlled by a horizontal synchronizing signal HD, a vertical synchronizing signal VD, etc. and synchronizes with each other. More specifically, the timing pulses which determine the operation timing of an electrical signal generation portion 2000 and the synchronizing signals which determine the operation timing of video signal producing portion 300 are generated in synchronization with each other.
Consequently, conversion of the optical image of the object into an electrical signal in electrical signal generation portion 2000 and conversion of the electrical signal from conversion portion 200 into a video signal in video signal producing portion 300 are conducted by appropriate timings.
FIG. 8 is a block diagram schematically showing the entire structure of a camera video tape recorder. Referring to FIG. 8, a description of the structure and operation of this conventional camcorder follows.
The camcorder includes an image sensing portion 100 for converting the optical image of an object into a video signal from which a video image can be reproduced on the picture frame of a light receiving surface, and a recording portion 400 for recording on a magnetic tape video signals Y and C produced at image sensing portion 100. Image sensing portion 100 includes the conversion portion 200 and video signal producing portion 300 of the conventional video camera (see FIG. 8). As is the case with the conventional video camera, the operation timings of conversion portion 200 and video signal producing portion 300 are controlled by timing generator 240 and synchronizing signal generator 250 which operate in synchronization with each other.
A luminance signal Y and a-chroma signal C produced by video signal producing portion 300 are input into an FM modulator 401 and a low band converter 402 at recording portion 400.
FM modulator 401 FM-modulates the luminance signal Y and applies the FM modulated luminance signal Y (hereinafter simply referred to as FM luminance signal) to a recording amplifier 403.
Low band converter 402 converts the frequency band of the chroma signal C into a prescribed low band which is lower than the frequency band of the FM luminance signal, and applies the converted chroma signal C (hereinafter simply referred to as low band chroma signal) to recording amplifier 403.
Recording amplifier 403 mixes the FM luminance signal and low band converted chroma signal and produces a video signal suitable for recording onto a magnetic tape 404. The video signal is recorded on the magnetic tape 404 by a head HW for recording.
As described above, the image sensed by image sensing portion 100 is recorded as the video signal on the magnetic tape 404.
A reading head HR reads the signal recorded on the magnetic tape 404 and applies the read signal to a reproducing amplifier 405.
Reproducing amplifier 405 amplifies the signal from the reading head HR and applies the amplified signal to a low pass filter 406 and a high pass filter 408.
Low pass filter 406 extracts the low frequency component of the signal output from reproducing amplifier 405, in other words extracts the low band converted chroma signal. The extracted low band converted chroma signal is applied to a high band converter 407.
High band converter 407 returns the frequency band of the low band converted chroma signal output from low pass filter 406 to the band it was in before the conversion by low band converter 402, and thus provides the original chroma signal C. The original chroma signal C is applied to a mixing amplifier 410.
High pass filter 408 extracts the high band component of the signal output from reproducing amplifier 405, in other words extracts the FM luminance signal, and applies the extracted FM luminance signal to an FM demodulator 409.
FM demodulator 409 demodulates the FM luminance signal from high pass filter 408 and provides the original luminance signal Y. The luminance signal Y is applied to mixing amplifier 410.
Mixing amplifier 410 mixes and amplifies the chroma signal C from high band converter 407 and the luminance signal Y from FM demodulator 409, and reproduces a video signal by which a video image can be reproduced on the picture plane of a TV receiver. The reproduced video signal is applied to the receiver through an output circuit 411.
The image sensed by image sensing portion 100 can therefore be provided in arbitrary time except for the time of sensing the image, by reproducing the video signal from the magnetic tape 404.
As in the foregoing, the conventional image sensing apparatus has only one optical system for taking up the optical image of an object (lenses 11 and iris 17, etc. in FIG. 7) and only one signal processing system (functional blocks succeeding image sensing device 12 in FIG. 7) for converting the optical image taken up into a video signal. It is therefore impossible for a single image sensing apparatus to reproduce an image on one picture plane using a plurality of video signal obtained by sensing the images of different objects at a time.
For example, in order to reproduce an image on one picture plane using two kinds of video signal obtained by sensing the images of two different kinds of objects at a time, two conventional image sensing apparatuses will be necessary. Furthermore, an additional circuit will be necessary for switching between or combining a video signal produced by one image sensing apparatus and a video signal produced by the other image sensing apparatus and applying the resultant signals to the receiver.
FIG. 9 is a block diagram schematically showing a structure which can readily be anticipated from such a point of view as a system for reproducing an image on one picture plane using two kinds of video signals obtained by sensing the images of different objects at a time. Referring to FIG. 9, this system includes two image sensing portions 100a and 100b having identical structures, an output circuit 4 for outputting a video signal to a receiver (not shown) and a switching circuit 3 for switching between the output signal of image sensing portion 100a and the output signal of image sensing portion 100b and applying the signal to output circuit 4.
Image sensing portions 100a and 100b each have an identical structure to a conventional image sensing apparatus shown in FIG. 7. In FIG. 9, in image sensing portion 100a, like functional blocks shown in FIG. 7 are designated by like numerals with the suffix "a" attached, while in image sensing portions 100b, like functional blocks shown in FIG. 7 are designated by like numerals with the suffix "b" attached. The operations of the functional blocks in image sensing portions 100a and 100b are the same as those in the case of the conventional image sensing apparatus shown in FIG. 7. A video signal corresponding to the optical image of the object captured by lenses 11a is therefore output from the adder 114a of image sensing portion 100a. Similarly, a video signal corresponding to the optical image of the object captured by lenses 11b is output from the adder 114b of image sensing portion 100b.
The video signal outputs from adders 114a and 114b are applied to output circuits 115a and 115b, respectively and also applied to switching circuit 3. Switching circuit 3 under the control of switching control circuit 5 selectively supplies either the video signal from adder 114a or the video signal from adder 114b to output circuit 4. Output circuit 4 externally outputs the video signal supplied from switching circuit 3.
In this system, a video signal output from either one of output circuits 4, 115a and 115b is applied to the receiver (not shown). If only the image of the object captured by lenses 11a is desired to be reproduced on the picture plane of the receiver, the output of the output circuit 115a is applied to the receiver. Similarly, if only the image of the object captured by lenses 11b is desired to be reproduced on the picture plane of the receiver, the output of output circuit 115b is applied to the receiver. If the image of the object captured by lenses 11a and the image of the object captured by lenses 11b are both desired to be reproduced on the picture plane of the receiver, the output signal of output circuit 4 is applied to the receiver.
Using switching circuit 3, it is possible, for example, to reproduce a combined image of the image of the object captured by lenses 11a and the image of the object captured by lenses 11b, or switch instantaneously the image displayed on the picture plane of the receiver from the object image captured by lenses 11a to the object image captured by lenses 11b or vice versa.
Switching control circuit 5, for example, generates a first signal which can control the internal connection state of switching circuit 3 so that switching circuit 3 applies a video signal from image sensing portion 100 to output circuit 4 and a second signal which can control the internal connection state of switching circuit 3 so that switching circuit 3 applies a video signal from image sensing portion 100b to output circuit 4. If the timing of switching a signal applied to switching circuit 3 from switching control circuit 5 from the first signal to the second signal and the timing of switching the signal from the second signal to the first signal can be somehow designated, an image produced by the image sensing of image sensing portion 100a and an image produced by the image sensing of the image sensing portion 100b can be displayed selectively or in combination on a single picture plane. When switching control circuit 5 continues to output the first signal and thus only the object image captured by lenses 11a is displayed on the entire picture plane of the receiver, the image on the entire picture plane can be switched to the image of the object captured by lenses 11b by switching the output signal of switching control circuit 5 to the second signal. Similarly, if switching control circuit 5 continues to output the second signal, and thus only the image of the object captured by lenses 11b is displayed on the entire picture plane, the image displayed on the picture plane can be instantaneously switched to the image of the object captured by lenses 11a by switching the output signal of switching control circuit 5 to the first signal. Furthermore, if the switching timing of the output signal of switching control circuit 5 is controlled so that the output signal of switching control circuit 5 is the first signal in the same prescribed period in any field period and the second signal in the other period, a part of the image of the object captured by lenses 11a and a part of the image of the object captured by lenses 11b are combined to be reproduced on a single picture plane.
FIG. 10 is a representation showing such an example of combination. Switching the output signal of switching control circuit 5 to the first signal (or the second signal) and the second signal (or the first signal) in the first half and second half of each field period, respectively allows the top half of the object image captured by lenses 11a (or 11b) and the bottom half of the object image captured by lenses 11b (or 11a) are combined and displayed. The switching timing of the output signal of switching control circuit 5 is controlled so that the output signal of switching control circuit 5 is the first signal (or the second signal) in a part of serial prescribed horizontal scanning periods in each field period, and is the second signal (or the first signal) in all the other horizontal scanning periods, and a combined image as shown in FIG. 4 is reproduced on the picture plane of the receiver. More specifically, the image of the object captured by lenses 11b (or 11a) is displayed on the entire picture plane with a part of the image replaced with a part of the object image captured by lenses 11a (or 11b).
Thus, using the conventional image sensing apparatus, the system which can process plural kinds of images produced by sensing the images of different objects at a time to be an image which can be displayed on the one picture plane requires optical systems and signal processing systems used for two conventional image sensing apparatuses. Such a system is on one hand highly functionable, processing a plurality of kinds of images produced by independent image sensing to be displayed on one picture plane, but on the other hand it is not practical in use because of the disadvantage in size reduction and cost reduction.
According to the conventional image sensing apparatus (see FIG. 7), it is not possible to instantaneously switch zoom magnifications, combine a plurality of object images which are sensed at different zoom magnifications and to expand the range of change of zoom magnifications too much.
Most of conventional image sensing apparatuses possess optical zoom function by which a remote object is sensed as if it is close by optically expanding the object. A general structure of lenses of an image sensing apparatus having such optical zoom function is shown in FIG. 11.
Referring to FIG. 11, the lenses generally includes a focus lens L1 for focusing an object, a zoom lens L2 for changing the focal distances of lenses, a main lens L3 for emitting light which have passed through focus lens L1 and zoom lens L1 upon the light receiving surface of the image sensing apparatus, which lenses are arranged in a optical axis X.
Focus lens L1 conducts focusing operation by its movement along the optical axis X.
The focal distances of lenses are continuously changed by the movement of zoom lens L2 along the optical axis. More specifically, the zoom magnification is determined uniformly depending upon the position of zoom lens L2 on the optical axis X. When image sensing is conducted under a constant zoom magnification, zoom lens L2 is fixed to a position from which a desired zoom magnification can be provided.
Therefore, for example, in FIG. 7, if lenses 11 include a zoom lens, the output signal of the adder 114 at the time of image sensing is a video signal obtained by sensing the image of the object at a prescribed zoom magnification.
If an object is image-sensed at a zoom magnification b (.noteq.a) after the object is initially image-sensed at a zoom magnification a, the zoom lens in lenses 11 is moved at a certain speed along the optical axis during the image sensing operation until the zoom magnification changes from a to b. At the time of image sensing operation as such, the output signal of adder 114 is not therefore switched instantaneously from a video signal corresponding to the object image sensed at the zoom magnification a to a video signal corresponding to the object image sensed at the zoom magnification b, but it becomes the video signal of the object image sensed at the zoom magnification b via the video signal of the object image sensed with the zoom magnification serially changed in the direction from a toward b. An image reproduced on the picture plane of the receiver by the output of output circuit 115 is therefore gradually switched from the object image magnified at the zoom magnification a to the object image magnified at the zoom magnification b.
In an image sensing apparatus having such optical zoom function, it will be necessary to use a lens with a large effective aperture diameter for the zoom lens for providing a high zoom magnification. The size of zoom lens should be increased in order to provide an image sensing apparatus having a high maximum zoom magnification. The image sensing apparatus having a high maximum zoom magnification which includes large-sized lenses 11 is therefore disadvantageous in terms of size reduction.
It is possible to implement instantaneous switching of zoom magnifications, and combining of the images of a plurality of objects sensed at different zoom magnifications by, for example, setting different values for the zoom magnification of lenses 11a and the zoom magnification of lenses 11b in the system having the structure shown in FIG. 9, but the system is encountered with the above-stated problems.
Furthermore, a conventional camcorder (see FIG. 8) includes single image sensing portion 100 which has only one signal processing system for converting the taken optical image into a video signal and only one optical system for taking up the optical image of an object, and a recording portion 400 for recording video signals Y and C produced at image sensing portion 100 on a magnetic tape. The conventional camcorder therefore can only record and reproduce the image of an object captured by one system of lenses. More specifically, the conventional camcorder cannot record a plurality of object images captured by a plurality of lens systems, or reproduce selectively these plurality of object images. In other words, the conventional image sensing apparatus with recording function cannot achieve various image sensing operations and various reproduction operations.
A construction of image sensing portion 100 as shown in FIG. 9 may be considered as one method of solving such a problem. However, such a method is still encountered with disadvantage in size reduction and cost reduction of the apparatus as stated above.